Real-Time Collaboration in a Hosted Word Processor

ABSTRACT

A computer-implemented method for coordinating documents between computer systems includes identifying, on a computing device, changes made by a user to a document having a model stored on the computing device, with corresponding models stored on a remote client and a server system; generating from the identified changes, one or more mutations that describe the changes and a one or more locations that correspond to positions in the model where the changes are to be provided; receiving one or more mutations from the server system and transforming the generated mutations using the received mutations; and updating the model stored on the computing device using the transformed mutations.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Application Ser.No. 61/323,275, filed on Apr. 12, 2010, entitled “Real-TimeCollaboration in a Hosted Word Processor,” the entire contents of whichare hereby incorporated by reference.

TECHNICAL FIELD

This document relates to systems and techniques for managing abrowser-based word processor or other form of document manager thatinteracts with a hosted server system to permit collaboration bymultiple users in a document.

BACKGROUND

Cloud-based, or hosted, computing generally involves executingapplications via a web browser, and obtaining information for theapplications from a remote server system or service. Variousapplications are available from hosted services, including searchengines, maps services, and productivity applications such as wordprocessor and spreadsheet programs that run by loading executing HTMLand JavaScript code from the server system to a browser running on aclient computing device that is remote from the server system.

Cloud computing provides real advantages over traditional desktopsoftware, such as the ability to access documents from various differentcomputers and locations. Web browsers do, however, place a number oflimits on programs that run on them. For example, web browsers may offerprogrammers HTML elements that are very simple to implement but can berather inflexible. As one example, a browser text box can be a simpleway for a programmer to acquire textual input from a user of a browser,but the manner in which the text is presented is relatively bland. Also,browsers do not provide access to native text rendering capabilities, donot generally permit much flexibility in positioning images on a webpage, and cannot natively execute other functions that are expected froma full-featured word processor application.

SUMMARY

This document describes systems and techniques that may be employed byusers of hosted word processing and similar applications to collaboratein the editing of documents. For example, as described here, a serversystem can share a model for a document with multiple client computingdevices, and the client computing devices can send changes, ormutations, made by their users to the server system regularly, such asmultiple times a second or for every keystroke or icon/menu selection.Each of the clients and the server may have complementary rules thatthey apply in order to resolve conflicts in overlapping edits, such aswhen one user edits part of a document at substantially the same timethat another user is deleting the entire phrase. In the techniquesdiscussed here, operational transformation features may be employed inbrowser-based word processors so that mutations or changes to a shareddocument can be made optimistically by client devices, but the servercan ensure that all versions of the document model converge toward onetrue version.

As described here, in general, each client device may cache informationregarding the locations in a document where the device has made changesto the model, and the nature of those changes. Such change informationmay be regularly uploaded to a server (e.g., multiple times per secondwhen the changes, such as keystrokes in a document, are occurring) thatis coordinating the word processing applications (or other applicationssuch as spreadsheet applications). Each client may maintain a list ofpending mutations that have not yet been sent to the server, and maytransform such mutations against any mutations that the server may sendto the client—e.g., that started at one of the other clients. As aresult, the client's list of pending mutations will always be valid forthe document state as of the client's last synchronization with theserver system. When a client does send change information up to theserver system, it may also send information that allows the serversystem to identify the client's last synchronization, so that the serversystem can transform the client's changes against all changes that haveoccurred since the last synchronization by the client.

Such features may, in certain implementations, provide one or moreadvantages. For example, a rich browser-based word processingapplication can be provided to users, so that they can receive thevarious benefits of cloud computing (e.g., access from a common browserinterface, access without needing to buy and/or download specialnon-browser application, and ability to share documents more easilyacross the internet) with the power and flexibility of traditionaldesktop applications (e.g., WYSIWYG presentation of document) and otherfeatures, such as real-time collaborative document editing amongmultiple users. Multiple users may also be able to use their clientsystems on the same document, so that wasteful individual editingsessions may be avoided, or conference calls in which users waste timeexplaining to a single person who has control of the document what to dowith the document's edits.

In one implementation, a computer-implemented method for coordinatingdocuments between computer systems is disclosed. The method comprisesidentifying, on a computing device, changes made by a user to a documenthaving a model stored on the computing device, with corresponding modelsstored on a remote client and a server system; generating from theidentified changes, one or more mutations that describe the changes anda one or more locations that correspond to positions in the model wherethe changes are to be provided; receiving one or more mutations from theserver system and transforming the generated mutations using thereceived mutations; and updating the model stored on the computingdevice using the transformed mutations. The model stored on thecomputing device can comprise a one-dimensional character string thatincludes content for the document, and a style map having pointers toindexed locations in the character string.

In another implementation, computer-implemented system for coordinatingelectronic document changes between computer systems is disclosed. Thesystem includes a document model stored on a first client computingdevice that defines content and formatting for an electronic document; alayout engine in communication with the document model and programmed torender the document model to a document object model (DOM) that isaccessible to a web browser that is executable on the first clientcomputing device; and a mutation resolver on the client computing deviceprogrammed to transform mutations made to the model on the first clientcomputing device that have not been transmitted to a server system,using mutations sent to the first client computing device by the serversystem, and then applying the transformed mutations to the model on thefirst client computing device.

In yet another implementation, a computer-implemented system forcoordinating electronic document changes between computer systemscomprises a document model stored on a first client computing devicethat defines content and formatting for an electronic document; a layoutengine in communication with the document model and programmed to renderthe document model to a document object model (DOM) that is accessibleto a web browser that is executable on the first client computingdevice; and means for resolving conflicting time-wise overlappingmutations made by the first client computing device and one or moreother client computing devices.

The details of one or more embodiments are set forth in the accompanyingdrawings and the description below. Other features and advantages willbe apparent from the description and drawings, and from the claims.

DESCRIPTION OF DRAWINGS

FIG. 1 is a conceptual diagram of a browser-based word processing systemfor collaborative editing of documents.

FIGS. 2A and 2B are block diagrams showing components of amodel-view-controller implementation of a spreadsheet application.

FIG. 2C is a block diagram of a system for permitting collaborativeediting of a documented by multiple users through a hosted serversystem.

FIG. 3A is a flow chart of an example client-side process forcollaborative editing of an electronic document.

FIG. 3B is a flow chart of an example server-side process forcollaborative editing of an electronic document.

FIG. 4 is an activity diagram showing status of two clients and a serverat various steps in a document collaboration process.

FIG. 5 shows an example of a computer device and a mobile computerdevice that can be used to implement the techniques described here.

Like reference symbols in the various drawings indicate like elements.

DETAILED DESCRIPTION

This document describes systems and techniques for browser-based wordprocessing applications. As discussed in more detail below, aclient-based application that runs in a web browser, such as using HTMLand JavaScript code, may download a portion of a document model from aserver system that hosts the online word processing functionality. Theapplication may create a user interface with the browser, and may placean editing surface on the display of the browser where the body of thedocument is to be displayed. The application may intercept keystrokesand other inputs by the user and may determine how the inputs are to bedisplayed by placing them in an off-screen or a highly minimized HTMLspan element, and measuring the size of such an element to determine thesize of the characters when they are properly displayed. The applicationmay also display a synthetic cursor on the editing surface and use themeasurements from the span element to determine the location at which toindex the cursor after the user has typed a character or made otherchanges to the document. As the user makes such changes, the changes maybe added to the local model, and may in turn be added to the display inthe browser by rendering the model or the changes to the model into adocument object model (DOM) that is accessed by the browser in afamiliar manner.

FIG. 1 is a conceptual diagram of a browser-based word processing system100 for collaborative editing of documents. In general, the system 100revolves around two users, named Chris and Spike, who are accessing andediting a document together in a collaborative manner. Each of the usershas a similarly configured client subsystem, because each hasindividually accessed a server system 104 and downloaded to theirbrowsers equivalent program code, such as in the form of HTML andJavaScript code. The two users in this example are currently viewing andediting the same document at the same time, and the system 100 is incharge of coordinating such activities by the two users.

The system 100 centers around individual copies of a document model 112,124, and efforts to optimistically update the copies of the documentmodel so that the server system 104 and the two users are experiencingthe same document, or are at least converging quickly (within seconds orless than a second) toward the same document. Rows of equal signs areshown in the figure between the document models 112 at the clients, andthe document model 124 at the server system 104, in order to emphasizethis effort to effectively match the document models with each otherwhen overlapping edits are being made to document model 112 at Chris'smachine, and document model 112 at Spike's machine.

The document models may be updated at each of the machines by applyingcertain transformation rules to pending changes, or mutations, that maybe applied to a model. For example, each of the clients may be forced totransform their own inputs to a document so that their changes reflectchanges received from other users via the server system 104, beforeapplying those changes to their own copy of the model 112. The varioussubcomponents in each of the client and server subsystems shown hereprovide examples of components that may achieve such coordinated editingof the document.

Referring now to particular components on the client devices, thesesubsystems are generally implemented as so-called model-view-controllerimplementations, where a model of a document may be maintained on aclient, the model may be rendered to create a view of the document modelwhich may cover all of a document model or a portion of the documentmodel, and a controller may intercept inputs made by a user with respectto the document, such as mouse clicks, keyboard and trees, and the like.The controller may then process such inputs to create changes to thedocument model, and those changes may then be reflected in the view whenthe updated document model is rendered to the view.

As shown here, each client computer communicates with the server system104 through an interface 120. The interface 120 may take various forms,but may include functionality for providing the server system 104 withrelevant information needed for the server system to maintaincoordination between the clients. Particular types of such informationare described more completely below. One type of information needed tomaintain coordination by the server system 104 is revisionidentification information from data store 122. Such information may beused to indicate the status of the most recent batch of mutations thathave been synchronized between the particular client and the serversystem 104, so that transformations of mutations may be applied in theappropriate direction (i.e., so that it can be determined whichtransformations “trump” other transformations) and appropriate batchesof mutations may be transformed against each other, whether at theserver system 104 or at the clients. As noted, mutations may be providedat the clients by various input mechanisms such as a keyboard 109, amouse, a touchscreen computer input, or other appropriate mechanisms.These inputs create the mutations that need to be coordinated by thesystem 100.

Such mutations or inputs may be captured or intercepted by a controller116. The controller 116 may use such inputs to identify and generatemutations corresponding to the user inputs. For example, if a user typesseveral characters at a particular location in a document, thecontroller 118 may generate a string that indicates the characters thatwere entered, and that recites a position in the model where suchcharacters are to be added to the model.

Each client may process individual mutations and submit individualmutations to the server system 104, such as by sending every keystrokereceived at the client computer up to server system 104 as a mutation.Alternatively, or at different times, multiple mutations may be gatheredtogether and stored as mutation lists 116, or mutations batches. Suchcaching of mutations may be beneficial, for example, when a user istyping very quickly, and a number of characters may be processedtogether to increase the efficiency of the system 100, while not slowingdown the responsiveness of the system 100. Thus, for example, when aclient device is quiet for a long time and a first mutation isidentified for the device (e.g., the user presses a key), a clientapplication may begin a timer, such as a timer set to expire in 200 msor a similar time, and may cache all mutations received during thattime, before processing and uploading the mutations as a batch.

A mutation resolver 114 on each client device may implement variousoperational transformation rules and techniques so that overlapping orinterfering mutations may be processed properly, and so that the models112, 124 in the system are all processed in a similar manner so thatthey can be expected to match after the processing is complete.Particular rules or techniques for resolving different batches ofmutations from different users are described in more detail below, suchas with respect to FIGS. 3A, 3B, and 4.

When a client has resolved particular mutations so as to determine whatchanges need to be made to a document model 112, the document model 112may be updated, such as by the controller 118. The display 106 of thedocument may then be updated so that the user at the appropriate clientcan see changes that he has made, and also changes that the other userhas made, in a relatively responsive and accurate manner. Suchgeneration of a new display 106 may occur using a layout engine 110 anda document object model (DOM) 108. The layout engine 110 may beprogrammed to process the document model 112 into the DOM 108. Thedocument model 112 may take a variety of forms, including aone-dimensional character string that includes the content of thedocument and that is supplemented by a sparse style map that representsformatting or styling for the characters and other items in the model112. The layout engine 110 may parse through the character string andapply formatting from the style map to generate HTML that syntheticallycreates a display 106 that looks or appears in a manner that the userintended. For example, each item in the display 106, such as eachcharacter in a written document, may be generated in an off screen spanelement using appropriate formatting, and the size of that element maybe determined using standard HTML elements. Each character that isprocessed in this manner may then be placed on an editing surface of thedisplay 106 at the appropriate location, and the location for the nextcharacter may be indexed forward by the width that was determined forthe previous character. A defined width (equivalent to page margins) maybe applied for each line, so that when the position of a character for aword exceeds that width, the position for all the characters of thatword may be shifted to the beginning of the next line (e.g.,implementing a word wrap feature). Also, a cursor may be shown on thedisplay 106 at an appropriate location as determined by the controller118, and may be a standard HTML element, such as a span element having anarrow width, and rendered in a background color that contrasts with thebackground color of the display 106, so that a user may locate thecursor easily.

The layout engine 112 can perform its processing in this manner, andgenerates, as an output, the DOM 108. Each user's browser may then usestandard techniques to convert the DOM on that particular client deviceinto display 106 for interacting with the relevant user.

Turning now to components used in the server system 104, there isprovided, in addition to the document model 124, a number of componentsthat are used to coordinate edits to a document made by user Chris anduser Spike. For example, a pending batches storage area 126 receivesmutation batches that have been submitted by the clients, and holds themuntil they may be processed. A batch processor or mutation resolver 128is responsible for applying operational transformations to the mutationbatches using rules or approaches like those discussed above and below.For example, a server system 104 may resolve a newly-received batch ofmutations from a client against all mutations that have been receivedsince the last time the client was synchronized with the server, so thatthe new mutations from that client do not take precedence over mutationshave been received previously from other clients.

A revision log 130 may be used in order to perform such comparisons. Inparticular, mutation batch is received from the various clients may bemaintained in the mutation log 130, which may include identifiers forparticular mutation batches, or revisions, received from each of theclients. These identifiers may then be used to identify or determine allof the mutations that have arrived since the previous synchronizationwith the particular client, and the identified mutations from the logmay have newly received mutations transformed against them, indetermining how the newly receive mutations are to affect the documentmodel 124. Particular techniques for applying such transformations aredescribed in more detail below with respect to FIGS. 3A, 3B, and 4.

We now discuss in more detail the arrangement of an examplemodel-view-controller client arrangement, and additional structures andother components that may be implemented in a collaborative documentediting system.

FIGS. 2A and 2B are block diagrams showing components of amodel-view-controller (MVC) implementation of a word processingapplication. The word processing application, for example, may beexecuted by a web browser, such as the browser 104 shown in FIG. 1A. Ingeneral, the MVC implementation provides for the download of a modelfrom a remote server to a client, and the rendering of the model into aDOM to form a view of the model that may be managed directly by the webbrowser. The controller may intercept actions, such as clicks on iconsand keystrokes on a keyboard, and may cause such actions to beimplemented, such as by adding typed characters both to the model on theclient and on the remote server (e.g., uploading changes or mutationsback to the server, which may be programmed with rules for integratingthe mutations into a master model, so that the client-side model matchesthe master model as the user works).

Referring to FIG. 2A, a computer application 200A may be configured todisplay a word processing document 202. The application 200A includes,controls, or accesses a model 204A, a view 206A, and a controller 208A.For example, the model 204A can contain a representation of the state ofthe word processing document 202, including such elements as characterdata, formats, styles, paragraphs, sections, breaks, lists, tables,images, formulas, and the like. The view 206A can represent a renderingof the current state of the model 204A. For example, the view canprovide a visual representation that combines the substance of thedocument (e.g., its raw text) with formatting and layout information.The view, when displayed by a browser, provides a form of, if not anexact, WYSIWYG representation of the document that is defined by themodel.

In addition to the rendering of the model, the view 206A can be used forpresenting to the user visual information that is associated with theword processing document 202, such as visible user controls for theapplication (i.e., chrome) and other word processing data. Thecontroller 208A can respond to changes in the model 204A or the view206A, and can update the state of the model 204A and the view 206A. Asshown in FIG. 2A, solid lines between the model 204A, the view 206A, andthe controller 208A represent direct references between components, anddashed lines represent listeners. For example, listening for userinteraction (e.g., provided by user controls) with the presentation ofthe view 206A, the controller 208A can modify the model 204A, and can inturn modify the view 206A either directly or indirectly (by causing theview 206A to obtain new model data and render that new data).

As another example, listening for changes in the model 204A made byanother user in a collaborative environment or changes made through anautomated data update or another such process, the view 206A can requestre-rendering of an updated model or portion of the model. For example,if a user of a client device is only one of multiple users concurrentlyediting a document, characters and other edits by the other users may bepassed to the client device from the server system (and edits by thefirst user may be passed from the client device to the server system),and the client code may add characters to the model in near real-time,and those changes can be passed into the DOM (e.g., via rendering by theview 206A)—so that each user can see the edits made by the other usersvery quickly.

Referring to FIG. 2B, the model-view-controller implementation aspresented in FIG. 2A is shown with additional detail. As shown, acomputer application 200B (corresponding with the application 200A)includes, controls, or accesses a model 204B (corresponding with themodel 204A), a view 206B (corresponding with the view 206A), and acontroller 208B (corresponding with the controller 208A).

The model 204B can include one or more document models 210. Each of thedocument models 210 can represent a separate document in a collection ofword processing documents, for example, and each of the models 210 caninclude elements such as characters, styles, and entities. Other formsof documents such as spreadsheet documents may also be represented.Model data and elements may be provided by a master document model 238that is stored on a remote server system via a connection to a network236 (e.g., the internet).

Generally, document text in the models 210 is associated with a seriesof characters. For example, the characters may represent raw text forthe word processing document 202, and may also include certain reservedcontrol characters such as characters that indicate the occurrence of abreak (e.g., a paragraph break, a page break, or the like). In someimplementations, each of the document models 210 can include aone-dimensional character string that includes document characters in anorder in which they appear in the document.

Styles may be used to store information related to the presentation ofdocument text (e.g., the series of characters). For example, text stylesmay include character formatting attributes such as font, font size,bold, italics, underline, foreground and background colors, alignment,and other such attributes. In some implementations, styles included ineach of the document models 210 can be stored in a sparse map. Forexample, the sparse map can include markers that correspond to changesin styles in the document and pointers to positions in the characterstring. The pointers, for example, can define style runs between matchedmarkers by specifying locations along the character string at whichstyle changes occur. In some implementations, the markers in the matchedset may be arranged to be tethered to certain characters in theone-dimensional character string. For example, if text is added betweentwo paired markers, the pointer for one of the markers may shift by aninteger equal to a number or characters associated with the added text.

Entities in each of the models 210 may be used to store informationrelated to objects outside of the document models 210, and may bepointed to by references in the model such as in the one-dimensionalcharacter string. For example, entities may include objects such aslists, tables, images, and the like. In some implementations, thereferences can include object identifiers and pointers to theone-dimensional character string (e.g., from the sparse map) indicatingwhere in the character string the object should appear, or identifiersfrom within the character string that point to the entities and mark thelocations at which the particular entities are to appear in the documentwhen it is rendered and displayed. For example, an image that shouldappear between two paragraphs may be associated with a special characterappearing in the character string after a paragraph marker for oneparagraph, and before the first character of the next paragraph.

The view 206B can generate one or more view items 220 that may enablethe user to interact with the application 200B, such as menu bars, toolbars, context menus, chat panes, dialogs, other chrome, and the like.The view 206B can also include a document menu 222 that presentsinformation and control options related to one or more of the documentmodels 210, and one or more of a set of per views 224. For example, oneof the per views 224 may be associated with a corresponding one of thesheet models 210. Each of the per views 224 may include components orcontrols such as selectors (e.g., cursors, selection indicators, and thelike) navigation tools (e.g., scrollbars, document maps, outlines, andthe like).

The controller 208B can include one or more controllers 230 that maylisten for and handle user interactions with one or more of the viewitems 220. In some implementations, each of the controllers 230 may beassociated with a corresponding one of the view items 220. For example,menu bar controllers may listen for and handle user interactions withmenu bar view items (e.g., relating to various actions that a user wouldtypically take from a row of menu selections), tool bar controllers maylisten for and handle user interactions with tool bar view items,context menu controllers may listen for and handle user interactionswith context menu view items, and so forth. The determination that aparticular event has occurred may cause a particular controller 230 toexecute predetermined code or otherwise carry out a predeterminedprocess, such as by updating a local model when a key press is receivedand uploading information about the key press to a central serversystem.

The controller 208B can also include a document controller 232 that maylisten for and handle user interactions with the document menu 222. Inaddition, the controller 208B can include a set of per view controllers234, where each of the controllers 234 is configured to listen for andhandle user interactions with a corresponding view in the set of perviews 224. Each of the per view controllers 234 may include variouscontroller types, such as key controllers for intercepting andinterpreting keyboard input, mouse controllers for intercepting andinterpreting mouse input, and model change controllers for interceptingand interpreting model change events.

Generally, the controllers included in the controller 208B can transformuser-generated events into model and view mutations. For example, basedon a user action, a relevant controller (e.g., a controller configuredfor handling the action) may receive one or more events associated withthe action and make transient changes to the view 206B before the useraction is committed. Then, based on the event properties, the relevantcontroller can construct a command to mutate the model 204B, execute it,and send the updated model or just data for the particular mutations tothe remote server system that hosts the document model 238 via thenetwork 236.

The controllers may also use timers or other mechanisms to aggregateinputs or mutations, so as to lower the number of updates that need tobe made to the local or server-based models. For example, thecontrollers may implement changes to the local and/or server-based modelin batches that occur within predefined time windows, such as by waiting200 ms after an initial keystroke is sensed before sending to thecentral server system data about all keystrokes received in the timewindow.

Many possible user interactions with the application 200B are possible,including interactions that are included in single-user sessions and inmultiple-user sessions. For purposes of illustration, a series ofexample user interactions with the application 200B are described here.For example, to enter text into the word processing document 202, theuser may proceed by using a computer mouse to select a desired documentlocation 212 for text insertion by clicking on the document 202. A mousecontroller that is included in the per view controllers 234 (e.g., a perview controller associated with the active document 202) can listen foran event that is associated with the mouse positioning input and themouse click input, and upon intercepting it, can modify the view 206B(e.g., a per view associated with the active document) to provide theuser with a visual indicator for the selection (e.g., a cursor). Forexample, the selected location 212 may be associated with a visiblecursor (where the cursor is a graphical HTML element such as an image,and is displayed at an appropriate location on a canvas where thedocument is also being displayed so as to create a syntheticpresentation that makes the graphical element look like a true cursor),may be highlighted, or may receive another such modification.Additionally, the selected location 212 may also be associated with alocation in the model 204B. For example, a position in a one-dimensionalcharacter string included in the document models 210 can be determined,based on the selected location 212.

Using a keyboard, the user may enter desired text at the documentlocation 212. A keyboard controller that is included in the per viewcontrollers 234 can listen for events associated with the keyboardinput, and upon intercepting them, can modify the view 206B to providethe user with a visual indicator for the input. For example, as the usertypes text at the location 212, the document 202 may be visually updatedto present the text to the user. Additionally, the keyboard controllercan modify the model 204B to include entered text by copying the userinput to the model. For example, a one-dimensional character stringincluded in the document models 210 can be updated to include theentered characters. Additionally, the document model 238 may be updatedto include the entered text, thus coordinating the model 204B with thedocument model 238. For example, changes to the model 204B may betransmitted to the document model 238 via a connection to the network236. In some implementations, changes may be sent periodically (e.g.,once every 100 milliseconds, once every 200 milliseconds, once every 500milliseconds, once every second, once every 2 seconds, or anotherappropriate time interval). In some implementations, changes may be sentbased on user activity (e.g., entering a paragraph break, applying aformatting change, navigating to another document section, clicking asave button, or some other action).

As another example, the user may insert an entity (e.g., a list, atable, a hyperlink, an image, or another such object) into the document202. For example, the user may proceed by using a computer mouse toselect a desired document location (e.g., the location 212) for entityinsertion by clicking on the document 202. Similar to interactionsassociated with text entry, for example, a mouse controller included inthe per view controllers 234 can listen for an event associated with themouse positioning input and the mouse click input, and upon interceptingit, can modify view 206B to provide the user with a visual indicator forthe selected location. Next, for example, the user may specify theentity for insertion by interacting with one of the view items 220 orwith the document menu 222. For example, the user may make a selectionon a menu bar to indicate an intent to insert an image. A dialogassociated with image selection may be presented to the user, enablingthe user to select the desired image.

Model-view-controller interactions for adding the entity within theapplication 200B may operate in a similar manner as when a user isentering text. For example, as the user inserts the image at thelocation 212, the document 202, as it is displayed on an editingsurface, may be visually updated to present the image to the user.Additionally, the model 204B may be modified to include a reference tothe inserted image by writing the reference to the model. For example,one of the document models 210 (e.g., the model associated with theactive document) can be updated to include a reference to the insertedimage. A one-dimensional character string may be updated to include aspecial character indicating the position of the image, and thereference to the image may be stored.

When the document 202 is rendered from the model associated with theactive document, for example, the image content may be integrated intothe document 202 that is displayed to the user. In some implementations,the one-dimensional character string may include multiple instances ofan identifier for a single entity. For example, the image may bepositioned at multiple locations in the document 202, specified bymultiple positions for the identifier in the one-dimensional characterstring. Thus, a single external entity may be shared within a document,or may be shared among multiple documents—both by common references tothe external entity. The document model 238 may be updated to includethe inserted image(s), thus coordinating the model 204B with thedocument model 238. For example, changes to the model 204B may betransmitted to the document model 238 via a connection to the network236.

Additionally, for example, the user may modify the formatting of textand entities presented in the document 202. By interacting with one ofthe view items 220 or with the document menu 222, the user can indicatea desired formatting change (e.g., a change such as changing a font of aselected text block to bold, changing a group of words to be organizedas a list, changing a paragraph justification to be right-aligned,changing a document line spacing to be double-spaced, and the like). Oneof the controllers 230 or the document controller 232 can listen foruser interaction with the view 206B, and upon detecting the interaction,can modify the model 204B to include the formatting change. For example,one of the document models 210 (e.g., the model associated with theactive document) can be updated to include an element in a sparse map ofstyles defining the formatting change, and defining the locations alongthe one-dimensional character string at which the changes in style areto occur. A model change controller included in the per view controllers234 can listen for events associated with the model 204B and can send arequest to the view 206B to update accordingly (e.g., by rendering adisplay of formatted text). In some implementations, the model changecontroller may also handle model change events that result fromcollaborative model changes.

In some implementations, a document model that is one of the documentmodels 210 may include a subset of the document data from the documentmodel 238. For example, if the document model 238 is substantiallylarge, a subset of the data (e.g., a subset associated with a portion ofthe document that is currently viewable by the user, plus perhaps abuffer area around the currently viewable area, or viewport, so thatsmall scrolling distances may be handled by normal browser interactionswithout a need to render additional content form the model) may beprovided to each client that is currently displaying the word processingdocument 202. As another example, the full document model 238 may beprovided to each client.

The user may elect to view a different portion of the word processingdocument 202 than the user is currently viewing in the web browser. Forexample, by interacting with a scrollbar 214 that is associated with thedocument 202, the user may indicate an intent to view document databeyond the current viewport, or displayed area. One of the per viewcontrollers 234 (e.g., the per view controller associated with theactive document) can listen for user interaction with the view 206B orother appropriate component (e.g., the visual portion of the scrollbar214), and upon detecting the interaction (e.g., via a computer mouse),can request for the view 206B to redraw itself.

If the user specifies a small amount of scrolling, the view 206A maycause itself to be displayed by the browser. For example, a buffer areaof document data may be maintained in the model 204B (already renderedinto a DOM) around the data that is displayed in the visible area of thedocument 202. If the amount of scrolling specified by the user isdetermined by the view 206B to be within the bounds of the buffer areaof data, the document display may be updated using such pre-rendereddata. If the user specifies a larger amount of scrolling, such that thescrolling specified by the user is determined by the view 206B to beoutside of the bounds of the pre-rendered buffer data, for example,additional document data from the document model 238 may be downloadedvia the network 236. Thus, the model 204B may be updated withinformation that is related to additional document sections, and thedocument may be rendered using the downloaded data.

FIG. 2C is a block diagram of a system 240 for permitting collaborativeediting of a document by multiple users through a hosted server system.In general, the system 240 includes a hosted document system 242executed by one or more computer servers (e.g. a server farm). Thehosted document system 242 can provide document hosting services to anynumber of client users via connections to a network 244 (e.g., theinternet). Using the document system 242, client users may create newdocuments, modify existing documents, share documents, andcollaboratively work on documents with other users.

For purposes of illustration, document hosting services may be providedto browser applications 246, 248, and 250. Each of the applications maybe executed by a web browser (e.g., by the browser 104, as shown in FIG.1), and may include model, view, and controller components (e.g.,similar to the application 200, shown in FIGS. 2A and 2B). Theapplications 246, 248, and 250 may be configured to execute computercode (e.g., JavaScript and other code running in a web browser) todisplay a word processing interface and to perform word processingfunctions associated with one or more documents served by the hosteddocument system 242.

As shown in the present illustration, Chris can interact with a webbrowser 252, Tina can interact with a web browser 254, and Spike caninteract with a web browser 256. Each of the browsers 252, 254, and 256may access any appropriate number of browser applications (e.g.,embedded applications, widgets, web services, and the like). Forexample, browser 252 can access application 246, browser 254 can accessapplication 248, and browser 256 can access application 250.

By interacting with controls presented by the web browsers, for example,users of the system 240 (e.g., Chris, Spike, and Tina), can work withone or more documents that are managed and provided by the hosteddocument system 242. For example, the users may access existingdocuments provided by the system 242 or may create new documents. Eachof the browser applications 246, 248, and 250 can communicate with aninterface 260 of the document system 242 via the network 244. Forexample, communication between the browser applications 246, 248, and250 and the interface 260 may include HTTP (HyperText Transfer Protocol)requests, SOAP (Simple Object Access Protocol) messages, or some otherappropriate such protocol. In some implementations, client browsers maymaintain browser channel connections to the interface 260 forcommunicating session data between clients and the document system 242.

The hosted document system 242 can include sub-components for storingand managing information related to system users, documents, and browserapplications. The various sub-components may be executed by the samecomputer server, or may be distributed among multiple computer servers.The sub-components may communicate with each other directly (e.g., viamessages, transferred files, shared data, remote procedure calls, orsome other protocol) or indirectly (e.g., by communicating with anintermediary application). Generally, sub-components included in thedocument system 242 can communicate with client applications (e.g., thebrowser applications 246, 248, and 250) via the interface 260.

The system 242 can also include one or more data stores for storing userinformation 270. For example, the user information 270 can includeinformation associated with system users (e.g., Chris, Tina, and Spike).Such information may include general user information and logininformation (e.g., user names, passwords, e-mail addresses, and thelike), information related to one or more devices employed by the usersto access the system (e.g., IP addresses, browser versions, connectionspeeds, and the like), and system usage information (e.g., access times,amount of data accessed, and the like), to name a few possibilities.

In some implementations, the system 242 can include one or more datastores for storing documents 272 in the form, e.g., of document modelslike those discussed above and below. For example, the documents 272 caninclude word processing documents created, maintained, and accessed bysystem users. As another example, the documents 272 may be generated byan automated process, such as a news feed or another reporting processthat is based on gathered data. Information associated with thedocuments 272 can include document data models, document text, documentformatting information, entities (e.g., tables, images, videos, soundclips, or other such objects), and the like.

The system 242 can also include one or more data stores for storingaccess information 274. For example, the access information 274 caninclude information that can be used for controlling access of systemusers (e.g., users included in the user information 270) to systemdocuments (e.g., documents included in the documents 272). Generally,system users may set access privileges for documents that they create ormanage. For example, Chris may create a personal letter document andspecify the document as being private. Thus, other users of the system(e.g., Tina and Spike) may be unable to locate or access the document,which may have access control limitations applied to it in variousfamiliar manners. As another example, Tina may upload a scheduledocument and specify the document as being shared and as being viewableby Chris. Thus, Spike may be unable to locate or access the document,but Chris may be able to access the document in view-only mode. In someimplementations, Tina, as the document creator, may retain full accessto the document, having privileges such as the ability to add, edit, anddelete content, having the ability to change privileges, and having theability to remove the document from the system 242. As another example,Spike may create a document related to a group project and specify Chrisand Tina (and himself) as having full access privileges. In someimplementations, user groups may be included in the access information274. For example, a user may create a group and may add one or moreusers to the group. Rather than select individual users when assigningdocument permissions, in some instances, users may select a groupincluding the users. The access information 274 may also include suchinformation as the user ids of document users, document access times,and the like.

In some implementations, the system 242 can include one or more datastores for storing HTML/JavaScript 276. For example, the HTML/JavaScript276 can include application code for executing the browser applications246, 248, and 250. The application code may be provided to any of thebrowsers 252, 254, and 256, for example, when browser users access a website associated with the hosted document system 242. Upon receiving arequest for any of the documents 272, for example, the system 242 mayprovide the HTML/JavaScript 276 in addition to one or more of thedocuments 272. Using the HTML/JavaScript 276, the browser applications246, 248, and 250 may render the document data and may provide aninterface that enables browser users to interact with the documents. Insome implementations, technologies other than HTML and JavaScript may beused for providing application code. For example, for web browsersincluding an appropriate plugin, another type of compiled or interpretedcode may be provided.

Many possible user interactions with the system 240 are possible,including interactions in single user sessions and in multiple usersessions. For example, in a collaborative editing session, multipleusers may simultaneously interact with a document. Although theapplications used for editing the document may each behaveindependently, the applications may follow the same editing rules forupdating and rendering the document model. Thus, multiple users may havesimilar experiences with the document, and may work together to producea similar document model.

In an example session, to initiate collaborative word processingdocument editing, Chris accesses the hosted document system 242 bydirecting the web browser 252 to a web site (e.g., a domain) that isassociated with the system 242. Receiving login information from thebrowser 252, the system 242 can verify Chris's information against theuser information 270. Upon verification, the system 242 can provideHTML/JavaScript 276 to the browser 252 for executing an online wordprocessor (though certain of the code may be passed before verificationoccurs). The browser can include a portion of the HTML/JavaScript 276 asthe browser application 246, render chrome associated with theapplication, and display the application to Chris.

Chris may interact with the browser application 246 via a set ofcontrols displayed in an application view within the browser 252. Forexample, Chris may indicate an intent to create a new document byclicking a button or selecting a menu option displayed in theapplication view. The application controller can intercept the commandand pass the command to the interface 260 via the network 244. Receivingthe command, the system 242 can add a new document to the documents 272,and add information associated with the new document to the set ofactive models 262. For example, the active models 262 may include modelinformation associated with documents currently being edited by otherusers of the system 242.

A corresponding version of a model in the set of active models 262 maybe present at the browser application 246. For example, Chris may addcontent and make changes to the word processing document provided by theview of the browser application 246, and the corresponding content andchanges can be applied to a model that is accessed by the browserapplication 246 (and associated HTML and JavaScript code running in thebrowser), and may be propagated to the active models 262.

Chris may also share the document with one or more users. For example,using controls associated with the application 246, Chris may selectTina and Spike as users who may share the document, and he may assignboth Tina and Spike full document privileges. For example, Tina andSpike may be included in a presented list of users commonly sharingdocuments with Chris, and Chris may select Tina and Spike from the list.As another example, Chris may provide the e-mail addresses of Tina andSpike. The system 242 can store the sharing information (e.g., user idsof other users having access to the document, permissions levels for theusers, and the like) in the access information 274. In someimplementations, the system 242 may send messages (e.g., e-mail, textmessages, instant messages, and the like) to users who have receiveddocument privileges. In some implementations, users who have receiveddocument privileges may receive a link (e.g., a hyperlink or URL) to theshared document.

Upon receiving notification of the shared document, Tina and Spike mayaccess the document using their web browsers 254, 256. For example, uponverification, the system 242 can provide HTML/JavaScript 276 to thebrowser 254, 256 for executing an online word processor. The browserscan include a portion of the HTML/JavaScript 276 as the browserapplications 248, 250, can render chrome associated with theapplication, and can display the applications.

Additionally, an active model manager 264 included the hosted documentsystem 242 can identify which documents are currently open by users ofthe system, and users who are active in the document (i.e., Chris), andcan set up a collaborative session. For example, the active modelmanager 264 can determine that the document requested by Tina and bySpike is associated with one or more of the active models 262. Thesystem 242 can then forward the document request to a computer hostingthe document, and the computer can associate Tina and Spike with thecurrent session. Additionally, the browser applications 248, 250 candownload model data associated with the active model(s) 262, and renderand display the downloaded model data. In some implementations, thesystem 242 can create model instances for Tina and for Spike and can addthe instances to the active models 262.

In the present example, users may be able to view their own cursors aswell as the cursors of other users in a collaborative session. Forpurposes of illustration, each user's cursor appears to himself/herselfas a square. For example, Chris may view his own cursor as a square, andthe other users' cursors as a circle or as a triangle. Correspondingly,Tina and Spike may also view their own cursor as a square, and the otherusers' cursors as circles or triangles. In some implementations, thecursors may appear as a different color (which could not be shown here).For example, cursors may generally appear as underlines or verticalbars, where the cursors are different colors for each user.

In the present example, changes made by each of the users can be sent bythe browser applications 246, 248, and 250 to the hosted document system242, coordinated, and sent back to the other users. In someimplementations, the changes can be sent at time intervals (e.g., onceevery 100 milliseconds, once every 200 milliseconds, once every 500milliseconds, once every second, once every 2 seconds, or anotherappropriate time interval). In some implementations, sending can bebased at least in part on user activity or inactivity. For example,during periods of user inactivity, changes may be sent or received lessfrequently than during periods of user activity. When a user is enteringdata or when a local user hovers over a cursor for another user, apop-up label that identifies the other user may be displayed, so thatthe local user can identify who is making changes—though the label maythen disappear so that it does not continue to block the document.

To coordinate multiple document changes made by multiple users, forexample, the hosted document system 242 can include collaboration logic266. For example, the collaboration logic 266 can be executed by one ormore code modules executed by one or more computer servers associatedwith the system 242. In some implementations, portions of thecollaboration logic can be executed by the browser applications 246,248, and 250. Generally, the logic 266 can resolve data collisions(e.g., instances where multiple users edit the same document portion orapply conflicting document formats) by applying a consistent set ofrules to all user changes. Although, in some instances, one or moreusers may be prompted to disambiguate a change. For example, if Tinamakes a document change and Spike makes a conflicting document changebefore receiving Tina's change, Spike may be presented with a messagefrom the browser application 250 including possible conflict resolutionscenarios. In some implementations, one user may be identified astrumping other users in collision situations. Chris, as the documentcreator, for example, may be able to apply his changes over changes madeby either Tina or Spike in cases of conflict. For example, if Spikeedits a passage at the same time as Chris deletes it, the passage(including Spike's edits) may be deleted.

Thus, the system shown in FIG. 2C may handle collaborative editing of ahosted document by multiple users at one time. The management of suchediting can involve a low amount of data passing between the varioussub-systems in the system

FIG. 3A is a flow chart of an example client-side process forcollaborative editing of an electronic document. In general, the processcenters around steps that are taken when he client-side applicationreceives input from a user of the application at essentially the sametime that another user working on another client is providing input tothe same document with a corresponding application at his or her clientdevice. The process here aims to maintain a single true document on bothclients and the server.

The process begins at box 300, where the client opens a browser channelto a server system. In this step, an initial communication session isopened so that subsequent communications may be instigated easily by theclient and by the server system. At box 302, the client downloads acompiled set of mutations made to the model, if the client currently hasan older version of the model, or may download a current version of themodel from scratch. When the mutations are received, the clientapplication may transform the stale version of the model using themutations, so that the now-current version of the model on the clientmatches a fresh version of the model on the server system.

At box 304, the client system begins receiving user edits to thedocument. For example, a user may position a cursor at a location in thedocument and begin typing text into the document, and deleting text fromthe document. Such interaction by the user may be implemented by amodel-view-controller arrangement like that described above. As oneexample, a controller may track the location of the cursor in thedocument, and may generate messages that describe mutations that theuser has made to the document. Those messages may, in particular,include information that identifies a particular location in thedocument model that corresponds to a location on a canvas at which themutations were made by the user.

At box 306, the client device updates its own local model to reflect thereceived mutations, and caches those mutations. The client device maybuild up a list of mutations in the cache to minimize the number oftransmissions that need to be sent to the server system. However, thetiming of the uploads may be set such that a user of the client deviceor another client device does not get too far out-of-sync with otherusers who are editing the document at the same time. At box 308, theclient device receives an update from the server system, which may be aset or batch of mutations that were previously submitted to the serversystem by another client device in the top been processed and forwardedfrom the court from the server system to the first client device. Thus,in this example, there are overlapping mutations in time that may theeffect of interfering with each other.

At box 310, the client device transforms its own mutations against thereceived update. Thus, as one simple example, if the client device hadpreviously recorded the inputs from its user as occurring in aparticular location in the document, and mutations received from theserver system were provided in an earlier location in the document, thelocation for the input provided on the client device may be shiftedbackward by an indexed amount that matches the number of characters inthe received mutations. Other transformations that follow from astandard operational transformation rule set may also be applied as isappropriate at this portion of the process. In implementations in whichthe first client has already placed its own mutations into a view of themodel, the view may be updated from the newly transformed model, such asby rendering the newly transformed model into a DOM that is accessibleto a web browser that the user is employing.

At box 312, the device determines that it has been collecting inputsfrom its user long enough, and triggers a batch shipment to the serversystem of mutations provided by its user. At box 314, those cachedmutations are sent to the server system. In addition, an identifier forthe last synchronization that the client had with the server system maybe sent. Such an identifier may be used by the server system so as toproperly set a baseline for transformations that it will need to performon the newly received mutations from the first client.

In addition, the first client may provide an identifier for itself, orthe server system may otherwise determine that the particular mutationshave arrived from that client. Such information may be useful inpermitting the server system to keep track of the source of variousmutations, and to link mutations to particular users, so that, forexample, the server system can make appropriate transformations tobatches of mutations that it receives and can forwarded transformedmutations to the appropriate other clients. Also, the server system maypass identifying information for a first client to other clients so thateach client application may show a tag next to newly entered or editedtext, where the tag is a pop-up that indicates the identity of user whoperformed the editing or entry. The user identifications may be stored,at least temporarily, in the various versions of the document model, sothat users can, for example, hover over a portion of text, and see apop-up box that indicates the identity of the user who provided thetext, and optionally a date stamp to indicate when the particular editoccurred.

At box 316, the client receives confirmation from the server system thatthe batch of mutations has been entered at the server system. Suchconfirmation may cause the client system to dump the pending mutationfrom a queue that tracks pending mutations, and also to index itsrevision number forward one position (box 318).

In this manner, a client device may operate optimistically in updatingits own version of a document model, and may also have some assurancethat its version of the document model will match, or will be matched,relatively quickly to other versions of the document model in thesystem. The end result is that users who are editing or operating in adocument in a collaborative fashion may quickly see changes that theyhave made and the other users have made, and interference between thechanges may be conveniently handled in a manner that maintains theintegrity of each of the versions of the document.

FIG. 3B is a flow chart of an example server-side process forcollaborative editing of an electronic document. In general, the processhighlights actions that may be performed on a central server system thatis coordinating edits made to a common document in a collaborativemanner by multiple users at different client computing devices.

The process begins at box 320, where the server system receives achannel request from one or more browsers. Upon receiving a request, theserver system may open the channel, and may serve to the requestingbrowser a list or group of compiled mutations. Such mutations may becalibrated to include all mutations that have occurred since theparticular client device was last provided mutations for updating of itsdocument model. The server system may then wait to receive informationfrom the various clients that are currently active in a session and mayremain idle during such a period.

At box 324, a communication is received from one of the clients. Thecommunication consists of a batch of mutations which may be termed “P”in this example, and an identifier for the last revision that the clienthas recorded as being synchronized between the client and the serversystem, which will be denoted “R” here. At box 326, the server systemappends the received batch from the particular client to a queue ofpending mutations. In certain implementations, where the server systemis able to process requests as fast as they arrive, a queue will not benecessary, or in certain situations, different queues may be maintainedfor each of the clients, and may be coordinated so that they areprocessed according to the time at which the particular request arrived.

At box 328, the server system creates a list, denoted as the letter “S”,of all mutations that arrived since the last batch of mutations from theparticular client have been processed, as indicated by the letter “R.”At box 330, the server system transforms the newly-received mutationsfrom the client against the mutations received from the group of clientssince time “R,” and such processing is used to create a group oftransformed mutations referenced here as P_(t). This newly createdentity is then appended to the server mutation log and is processed indue course by the server system (box 332).

The server system may then send communications to each of the clientsregarding the received mutations. For example, at box 336, the serversystem sends a confirmation message back to the originating client forthe batch of vacations, so that the originating client may know that itsmutations were received and processed, and may index its revisiontracking number forward a position. At box 334, in contrast, the serversystem may send information to the other client so that they mayimplement the mutations received from the first client in appropriatemanners. To do so, the server system may send the P_(t) batch to all theclients other than the submitting client, a revision number for thatbatch, an identification of the originating client for the mutations,and a current time, where such information is appropriate. Each of theother clients may then process the received information in anappropriate manner, such as by using mechanisms described above,including the process described in FIG. 3A.

FIG. 4 is an activity diagram showing status of two clients and a serverat various steps in a document collaboration process. In general, theprocess shows how text typed by the two different users of the twoclients may be propagated through the server and passed to the otherclient. In the top row, each of the sub-systems is simply initialized.Each client may track the number of revisions it has sent to the server,and that number may initially set to zero. The sent and unsent queuesfor the client application (in the middle of each box for both clients)are also empty, and the output of the client-stored models is an emptyset. Likewise, the server system has no document to display yet, andalso nothing to place in log or batches storage.

After the first row of boxes in time, user Chris types the message“Hello”, which appears in the document on Chris's device, and isreformatted into a mutation of the form: insert @1 “Hello”. Thatmutation then enters Chris' unsent queue for a short period, such as asmall fraction of a second, and is then transmitted to the to the serversystem, along with an indication that the revision number is 0. Thedevice for Chris also updates its queues. Specifically, to show that thefirst revision has been sent. Through this time, user Spike has not beeninteracting with his word processing application, so both the serversystem and Spike's client have been idle through this period.

In the third row then, Chris's queues have been updated, and the servernow has a batch to be processed that includes the mutation, the revisionnumber passed by Chris's client device, and an identifier to indicatethat the edits came from Chris.

Between rows three and four, Chris has continued typing and the serversystem has processed the submission of the first mutation (or moreproperly, a batch of five mutations) by transforming the receivedmutations against its own version of the model. Thus, by row four, theoriginal mutation from Chris's device has moved into the log for theserver system and has been assigned a logging number. At the same timethe unsent queue on Chris's machine has received the term “world” fromChris's continued typing on his client device. Again, Spike has remainedquiet

At row five, a couple interesting actions occur. First, the serversystem recognizes the presence of the first revision in its log, andsends a confirmation message back to Chris's device. Second, Chris'ssecond set of mutations has arrived at the server system and is waitingto be processed. Third, Spike has begun typing at his device, and hasentered the letters “foo”. Thus, after the row of boxes in the fifthrow, multiple messages are being sent, including an upload of Spike'sfirst revision, a confirmation of Chris's first revision, and areporting to Spike of Chris's first revision.

In the sixth row of boxes, Chris's device has incremented its revisionnumber as a result of receiving the confirmation, so that it canproperly track and compare mutations or batches of mutations whenmutations are received from Spike via the server system. Also, theserver system has receive Spike's mutations and is processing Chris'ssecond set of mutations. And Spike's computer is transforming itspending mutations against the mutations from Chris that were justreceived from the server system. In this manner, Spike's computer willbe performing the same mutations performed by the server system, andwill not give his own entry prominence over Chris's first entry. Theprocess, of course, may continue as each of Chris and Spike enter andedit text in their own copies of the document model, and the variousdocument models are processed so as to bring them back into line witheach other.

FIG. 4 shows an example of a generic computer device 400 and a genericmobile computer device 450, which may be used with the techniquesdescribed here. Computing device 400 is intended to represent variousforms of digital computers, such as laptops, desktops, workstations,personal digital assistants, servers, blade servers, mainframes, andother appropriate computers. Computing device 450 is intended torepresent various forms of mobile devices, such as personal digitalassistants, cellular telephones, smartphones, and other similarcomputing devices. The components shown here, their connections andrelationships, and their functions, are meant to be exemplary only, andare not meant to limit implementations of the inventions describedand/or claimed in this document.

Computing device 400 includes a processor 402, memory 404, a storagedevice 406, a high-speed interface 408 connecting to memory 404 andhigh-speed expansion ports 410, and a low speed interface 412 connectingto low speed bus 414 and storage device 406. Each of the components 402,404, 406, 408, 410, and 412, are interconnected using various busses,and may be mounted on a common motherboard or in other manners asappropriate. The processor 402 can process instructions for executionwithin the computing device 400, including instructions stored in thememory 404 or on the storage device 406 to display graphical informationfor a GUI on an external input/output device, such as display 416coupled to high speed interface 408. In other implementations, multipleprocessors and/or multiple buses may be used, as appropriate, along withmultiple memories and types of memory. Also, multiple computing devices400 may be connected, with each device providing portions of thenecessary operations (e.g., as a server bank, a group of blade servers,or a multi-processor system).

The memory 404 stores information within the computing device 400. Inone implementation, the memory 404 is a volatile memory unit or units.In another implementation, the memory 404 is a non-volatile memory unitor units. The memory 404 may also be another form of computer-readablemedium, such as a magnetic or optical disk.

The storage device 406 is capable of providing mass storage for thecomputing device 400. In one implementation, the storage device 406 maybe or contain a computer-readable medium, such as a floppy disk device,a hard disk device, an optical disk device, or a tape device, a flashmemory or other similar solid state memory device, or an array ofdevices, including devices in a storage area network or otherconfigurations. A computer program product can be tangibly embodied inan information carrier. The computer program product may also containinstructions that, when executed, perform one or more methods, such asthose described above. The information carrier is a computer- ormachine-readable medium, such as the memory 404, the storage device 406,memory on processor 402, or a propagated signal.

The high speed controller 408 manages bandwidth-intensive operations forthe computing device 400, while the low speed controller 412 manageslower bandwidth-intensive operations. Such allocation of functions isexemplary only. In one implementation, the high-speed controller 408 iscoupled to memory 404, display 416 (e.g., through a graphics processoror accelerator), and to high-speed expansion ports 410, which may acceptvarious expansion cards (not shown). In the implementation, low-speedcontroller 412 is coupled to storage device 406 and low-speed expansionport 414. The low-speed expansion port, which may include variouscommunication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet)may be coupled to one or more input/output devices, such as a keyboard,a pointing device, a scanner, or a networking device such as a switch orrouter, e.g., through a network adapter.

The computing device 400 may be implemented in a number of differentforms, as shown in the figure. For example, it may be implemented as astandard server 420, or multiple times in a group of such servers. Itmay also be implemented as part of a rack server system 424. Inaddition, it may be implemented in a personal computer such as a laptopcomputer 422. Alternatively, components from computing device 400 may becombined with other components in a mobile device (not shown), such asdevice 450. Each of such devices may contain one or more of computingdevice 400, 450, and an entire system may be made up of multiplecomputing devices 400, 450 communicating with each other.

Computing device 450 includes a processor 452, memory 464, aninput/output device such as a display 454, a communication interface466, and a transceiver 468, among other components. The device 450 mayalso be provided with a storage device, such as a microdrive or otherdevice, to provide additional storage. Each of the components 450, 452,464, 454, 466, and 468, are interconnected using various buses, andseveral of the components may be mounted on a common motherboard or inother manners as appropriate.

The processor 452 can execute instructions within the computing device450, including instructions stored in the memory 464. The processor maybe implemented as a chipset of chips that include separate and multipleanalog and digital processors. The processor may provide, for example,for coordination of the other components of the device 450, such ascontrol of user interfaces, applications run by device 450, and wirelesscommunication by device 450.

Processor 452 may communicate with a user through control interface 458and display interface 456 coupled to a display 454. The display 454 maybe, for example, a TFT LCD (Thin-Film-Transistor Liquid Crystal Display)or an OLED (Organic Light Emitting Diode) display, or other appropriatedisplay technology. The display interface 456 may comprise appropriatecircuitry for driving the display 454 to present graphical and otherinformation to a user. The control interface 458 may receive commandsfrom a user and convert them for submission to the processor 452. Inaddition, an external interface 462 may be provide in communication withprocessor 452, so as to enable near area communication of device 450with other devices. External interface 462 may provide, for example, forwired communication in some implementations, or for wirelesscommunication in other implementations, and multiple interfaces may alsobe used.

The memory 464 stores information within the computing device 450. Thememory 464 can be implemented as one or more of a computer-readablemedium or media, a volatile memory unit or units, or a non-volatilememory unit or units. Expansion memory 474 may also be provided andconnected to device 450 through expansion interface 472, which mayinclude, for example, a SIMM (Single In Line Memory Module) cardinterface. Such expansion memory 474 may provide extra storage space fordevice 450, or may also store applications or other information fordevice 450. Specifically, expansion memory 474 may include instructionsto carry out or supplement the processes described above, and mayinclude secure information also. Thus, for example, expansion memory 474may be provide as a security module for device 450, and may beprogrammed with instructions that permit secure use of device 450. Inaddition, secure applications may be provided via the SIMM cards, alongwith additional information, such as placing identifying information onthe SIMM card in a non-hackable manner.

The memory may include, for example, flash memory and/or NVRAM memory,as discussed below. In one implementation, a computer program product istangibly embodied in an information carrier. The computer programproduct contains instructions that, when executed, perform one or moremethods, such as those described above. The information carrier is acomputer- or machine-readable medium, such as the memory 464, expansionmemory 474, memory on processor 452, or a propagated signal that may bereceived, for example, over transceiver 468 or external interface 462.

Device 450 may communicate wirelessly through communication interface466, which may include digital signal processing circuitry wherenecessary. Communication interface 466 may provide for communicationsunder various modes or protocols, such as GSM voice calls, SMS, EMS, orMMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS, among others.Such communication may occur, for example, through radio-frequencytransceiver 468. In addition, short-range communication may occur, suchas using a Bluetooth, WiFi, or other such transceiver (not shown). Inaddition, GPS (Global Positioning System) receiver module 470 mayprovide additional navigation- and location-related wireless data todevice 450, which may be used as appropriate by applications running ondevice 450.

Device 450 may also communicate audibly using audio codec 460, which mayreceive spoken information from a user and convert it to usable digitalinformation. Audio codec 460 may likewise generate audible sound for auser, such as through a speaker, e.g., in a handset of device 450. Suchsound may include sound from voice telephone calls, may include recordedsound (e.g., voice messages, music files, etc.) and may also includesound generated by applications operating on device 450.

The computing device 450 may be implemented in a number of differentforms, as shown in the figure. For example, it may be implemented as acellular telephone 480. It may also be implemented as part of asmartphone 482, personal digital assistant, or other similar mobiledevice.

Various implementations of the systems and techniques described here canbe realized in digital electronic circuitry, integrated circuitry,specially designed ASICs (application specific integrated circuits),computer hardware, firmware, software, and/or combinations thereof.These various implementations can include implementation in one or morecomputer programs that are executable and/or interpretable on aprogrammable system including at least one programmable processor, whichmay be special or general purpose, coupled to receive data andinstructions from, and to transmit data and instructions to, a storagesystem, at least one input device, and at least one output device.

These computer programs (also known as programs, software, softwareapplications or code) include machine instructions for a programmableprocessor, and can be implemented in a high-level procedural and/orobject-oriented programming language, and/or in assembly/machinelanguage. As used herein, the terms “machine-readable medium”“computer-readable medium” refers to any computer program product,apparatus and/or device (e.g., magnetic discs, optical disks, memory,Programmable Logic Devices (PLDs)) used to provide machine instructionsand/or data to a programmable processor, including a machine-readablemedium that receives machine instructions as a machine-readable signal.The term “machine-readable signal” refers to any signal used to providemachine instructions and/or data to a programmable processor.

To provide for interaction with a user, the systems and techniquesdescribed here can be implemented on a computer having a display device(e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor)for displaying information to the user and a keyboard and a pointingdevice (e.g., a mouse or a trackball) by which the user can provideinput to the computer. Other kinds of devices can be used to provide forinteraction with a user as well; for example, feedback provided to theuser can be any form of sensory feedback (e.g., visual feedback,auditory feedback, or tactile feedback); and input from the user can bereceived in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in acomputing system that includes a back end component (e.g., as a dataserver), or that includes a middleware component (e.g., an applicationserver), or that includes a front end component (e.g., a client computerhaving a graphical user interface or a Web browser through which a usercan interact with an implementation of the systems and techniquesdescribed here), or any combination of such back end, middleware, orfront end components. The components of the system can be interconnectedby any form or medium of digital data communication (e.g., acommunication network). Examples of communication networks include alocal area network (“LAN”), a wide area network (“WAN”), and theInternet.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other.

A number of embodiments have been described. Nevertheless, it will beunderstood that various modifications may be made without departing fromthe spirit and scope of the invention. In addition, the logic flowsdepicted in the figures do not require the particular order shown, orsequential order, to achieve desirable results. In addition, other stepsmay be provided, or steps may be eliminated, from the described flows,and other components may be added to, or removed from, the describedsystems. Accordingly, other embodiments are within the scope of thefollowing claims.

1. A computer-implemented method for coordinating documents betweencomputer systems, the method comprising: identifying, on a computingdevice, changes made by a user to a document having a model stored onthe computing device, with corresponding models stored on a remoteclient and a remote server system; generating from the identifiedchanges, one or more mutations that describe the changes and a one ormore locations that correspond to positions in the model where thechanges are to be provided; receiving one or more mutations from theserver system and transforming the generated mutations using thereceived mutations; and updating the model stored on the computingdevice using the transformed mutations.
 2. The method of claim 1,wherein the model stored on the computing device comprises aone-dimensional character string that includes content for the document,and a style map having pointers to indexed locations in the characterstring.
 3. The method of claim 1, wherein updating the model comprisesapplying an updating rule set that is complementary to updating rulessets at the remote client and the remote server system.
 4. The method ofclaim 1, further comprising periodically receiving a model reset signaland an updated model definition from the remote server system, andreplacing a current model on the computing device with the updated modeldefinition.
 5. The method of claim 1, further comprising cachingmutations at the computing device and, upon receiving one or moremutations form the server system, applying conflict resolution rulesbetween the cached mutations and the received mutations.
 6. The methodof claim 1, further comprising rendering the updated model by generatinga document object model (DOM) from the updated model.
 7. Acomputer-implemented system for coordinating electronic document changesbetween computer systems, comprising: a document model stored on a firstclient computing device that defines content and formatting for anelectronic document; a layout engine in communication with the documentmodel and programmed to render the document model to a document objectmodel (DOM) that is accessible to a web browser that is executable onthe first client computing device; and a mutation resolver on the clientcomputing device programmed to transform mutations made to the model onthe first client computing device that have not been transmitted to aserver system, using mutations sent to the first client computing deviceby the server system, and then applying the transformed mutations to themodel on the first client computing device.
 8. The system of claim 7,wherein the document model stored on the first client computing devicecomprises a one-dimensional character string that includes content for adocument, and a style map having pointers to indexed locations in thecharacter string.
 9. The system of claim 7, wherein the first clientcomputing device is programmed to update the model by applying anupdating rule set that is complementary to updating rules sets at theremote client and the remote server system.
 10. The system of claim 7,further comprising a server system programmed to periodically send to aplurality of remote computing devices, including the first clientcomputing device, a model reset signal and an updated model definitionfor the computing devices to use in refreshing document models at eachof the computing devices.
 11. The system of claim 7, further comprisinga mutation cache storing mutations waiting to be uploaded to a serversystem, and wherein the mutation resolver operates on mutations receivedfrom the server system by comparing them to mutations in the mutationcache.
 12. The system of claim 7, wherein the first client computingdevice is programmed to render an updated model by generating a documentobject model (DOM) from an updated model.
 13. A computer-implementedsystem for coordinating electronic document changes between computersystems, comprising: a document model stored on a first client computingdevice that defines content and formatting for an electronic document; alayout engine in communication with the document model and programmed torender the document model to a document object model (DOM) that isaccessible to a web browser that is executable on the first clientcomputing device; and means for resolving conflicting time-wiseoverlapping mutations made by the first client computing device and oneor more other client computing devices.
 14. One or more tangiblecomputer-readable media storing instructions that, when executed, causeone or more computers to perform operations comprising: identifying, ona computing device, changes made by a user to a document having a modelstored on the computing device, with corresponding models stored on aremote client and a remote server system; generating from the identifiedchanges, one or more mutations that describe the changes and a one ormore locations that correspond to positions in the model where thechanges are to be provided; receiving one or more mutations from theserver system and transforming the generated mutations using thereceived mutations; and updating the model stored on the computingdevice using the transformed mutations.
 15. The tangiblecomputer-readable media of claim 14, wherein the model stored on thecomputing device comprises a one-dimensional character string thatincludes content for the document, and a style map having pointers toindexed locations in the character string.
 16. The tangiblecomputer-readable media of claim 14, wherein updating the modelcomprises applying an updating rule set that is complementary toupdating rules sets at the remote client and the remote server system.17. The tangible computer-readable media of claim 14, wherein theoperations further comprise periodically receiving a model reset signaland an updated model definition from the remote server system, andreplacing a current model on the computing device with the updated modeldefinition.
 18. The tangible computer-readable media of claim 14,wherein the operations further comprise caching mutations at thecomputing device and, upon receiving one or more mutations form theserver system, applying conflict resolution rules between the cachedmutations and the received mutations.
 19. The tangible computer-readablemedia of claim 1, wherein the operations further comprises rendering theupdated model by generating a document object model (DOM) from theupdated model.